The structured elastic net for quantile regression and support vector classification

In view of its ongoing importance for a variety of practical applications, feature selection via `1-regularization methods like the lasso has been subject to extensive theoretical as well empirical investigations. Despite its popularity, mere `1-regularization has been criticized for being inadequate or ineffective, notably in situations in which additional structural knowledge about the predictors should be taken into account. This has stimulated the development of either systematically different regularization methods or double regularization approaches which combine `1-regularization with a second kind of regularization designed to capture additional problem-specific structure. One instance thereof is the ’structured elastic net’, a generalization of the proposal in Zou and Hastie [2005], studied in Slawski et al. [2010] for the class of generalized linear models. In this paper, we elaborate on the structured elastic net regularizer in conjunction with two important loss functions, the check loss of quantile regression and the hinge loss of support vector classification. Solution paths algorithms are developed which compute the whole range of solutions as one regularization parameter varies and the second one is kept fixed. The methodology and practical performance of our approach is illustrated by means of case studies from image classification and climate science. keywords: double regularization, elastic net, quantile regression, solution path, support vector classification